1. Field of the Invention
The present invention relates to an image capturing apparatus for obtaining an image captured by an imaging optical system and an image processing method that reduces deterioration of the image quality of the image due to optical characteristics of the imaging optical system by an image recovery operation.
2. Description of the Related Art
In the case of capturing an image of an object through an imaging optical system, there are cases where the captured image exhibits some blurring or has undesirable coloring (hereinafter referred to as “aberration”) due to aberration of the imaging optical system, defocusing, light diffraction, and so on. In response, an image processing technique has been developed in which such blurring and aberration are eliminated from the captured image, using optical characteristics (for example, a point spread function, which is hereinafter referred to as a “PSF”) of the imaging optical system (see Japanese Patent Laid-Open No. 2007-183842, for example). Such image processing is hereinafter referred to as “image recovery”.
Now, the principles of image recovery will be described in short. If f is a captured image and noise is sufficiently small, f is obtained by a convolution of an image d obtained after recovery (hereinafter referred to as a “recovered image”) and a PSF p, as seen from the following Equation (1). The asterisk * in Equation (1) represents a convolution operation.f=d*p  (1)
In general, although p depends on the position in the captured image, Equation (1) can be approximated to the following Equation (2) by a Fourier transform in a range where dependency is considered to be low.F=D×P  (2)
In Equation (2), F, D, and P are spectrums of f, d, and p, respectively, which are obtained by a Fourier transform, and in particular, P is referred to as an optical transfer function (OTF). Also, x represents a computation for obtaining a product at each point in the image.
Here, f can be obtained by image capture, and thus if p is known, then image recovery can be performed as follows.
In the case of performing an image recovery operation in frequency space, Equation (3) is obtained by dividing F by P in Equation (2), and D is obtained from the known F and P.D=F/P  (3)
The recovered image d can thus be obtained by performing an inverse Fourier transform of D obtained from Equation (3).
In the case of performing an image recovery operation in real space, Equation (3) is transformed into the following Equation (4), and both sides of Equation (4) are subjected to an inverse Fourier transform, so that Equation (5) is obtained.D=F×(1/P)  (4)d=f*r  (5)
Note that, in Equation (5), r is obtained by an inverse Fourier transformation of (1/P), and the resultant recovered image d equals a convolution of the captured image f and the value r.
As described above, in order to recover a captured image from blurring or aberration properly, optical characteristics (such as a PSF) in the imaging optical system need to be known. In general, optical characteristics are considered to be determined by optical design parameters of the optical system that include an F number and a focal length, zooming, and so on during image capture.
Meanwhile, a technique has also been disclosed in which the amount of recovery during image recovery using optical characteristics is controlled by introducing parameters that change the amount of recovery continuously from “no recovery” to “recovery as specified by optical design parameters” (see Japanese Patent Laid-Open No. 2007-183842, for example).
However, optical characteristics of the imaging optical system during actual image capture are determined by various factors that cannot be determined by only optical design parameters. Conceivable examples of such factors determining optical characteristics include the spectrum bias of an object, and defocusing of a photographer's intended object. If optical characteristics during actual image capture differ from those calculated from only optical design parameters (hereinafter referred to as “design optical characteristics”) because of such factors determining optical characteristics (hereinafter, factors), even if recovery processing is performed using the design optical characteristics, the image cannot be recovered properly. Furthermore, if there is a large discrepancy, that is, a large error, between varying optical characteristics during actual image capture and design optical characteristics, there is a problem in that image quality may instead deteriorate by performing recovery processing using the design optical characteristics.
The present invention has been developed in light of the above-described problems, and aims to provide an image capturing apparatus that obtains a favorable recovered image by performing image recovery taking into consideration variations in optical characteristics of an imaging optical system, and to provide an image processing method.